Open-Vocabulary Semantic Parsing with both Distributional Statistics and Formal Knowledge

Traditional semantic parsers map language onto compositional, executable queries in a fixed schema. This mapping allows them to effectively leverage the information contained in large, formal knowledge bases (KBs, e.g., Freebase) to answer questions, but it is also fundamentally limiting---these semantic parsers can only assign meaning to language that falls within the KB's manually-produced schema. Recently proposed methods for open vocabulary semantic parsing overcome this limitation by learning execution models for arbitrary language, essentially using a text corpus as a kind of knowledge base. However, all prior approaches to open vocabulary semantic parsing replace a formal KB with textual information, making no use of the KB in their models. We show how to combine the disparate representations used by these two approaches, presenting for the first time a semantic parser that (1) produces compositional, executable representations of language, (2) can successfully leverage the information contained in both a formal KB and a large corpus, and (3) is not limited to the schema of the underlying KB. We demonstrate significantly improved performance over state-of-the-art baselines on an open-domain natural language question answering task.Comment: Re-written abstract and intro, other minor changes throughout. This version published at AAAI 201

Simple and Effective Multi-Paragraph Reading Comprehension

We consider the problem of adapting neural paragraph-level question answering models to the case where entire documents are given as input. Our proposed solution trains models to produce well calibrated confidence scores for their results on individual paragraphs. We sample multiple paragraphs from the documents during training, and use a shared-normalization training objective that encourages the model to produce globally correct output. We combine this method with a state-of-the-art pipeline for training models on document QA data. Experiments demonstrate strong performance on several document QA datasets. Overall, we are able to achieve a score of 71.3 F1 on the web portion of TriviaQA, a large improvement from the 56.7 F1 of the previous best system.Comment: 11 pages, updated a referenc

Crowdsourcing Multiple Choice Science Questions

We present a novel method for obtaining high-quality, domain-targeted multiple choice questions from crowd workers. Generating these questions can be difficult without trading away originality, relevance or diversity in the answer options. Our method addresses these problems by leveraging a large corpus of domain-specific text and a small set of existing questions. It produces model suggestions for document selection and answer distractor choice which aid the human question generation process. With this method we have assembled SciQ, a dataset of 13.7K multiple choice science exam questions (Dataset available at http://allenai.org/data.html). We demonstrate that the method produces in-domain questions by providing an analysis of this new dataset and by showing that humans cannot distinguish the crowdsourced questions from original questions. When using SciQ as additional training data to existing questions, we observe accuracy improvements on real science exams.Comment: accepted for the Workshop on Noisy User-generated Text (W-NUT) 201

The Phonology of the Canadian Shift Revisited: Thunder Bay & Cape Breton

Previous accounts of the Canadian Shift, which have interpreted this diachronic process as a purely phonetic consequence of the low back LOT-THOUGHT vowel merger, have not clearly explained the strong connection between phonetic TRAP vowel retraction and the phonological process of the low back merger. This paper addresses this issue in several ways. Relying on the Modified Contrastive Specification theory (Dresher et al. 1994) and the Contrastive Hierarchy approach (Dresher 2009), two phonological frameworks, as well as phonetic insights from Vowel Dispersion theory (Liljencrants and Lindblom 1972) and Dispersion-Focalization theory (Schwartz et al. 1997, Schwartz et al. 2007), we propose that the catalyst of the Canadian Shift is a three-way merger of the PALM, LOT and THOUGHT lexical sets, in combination with a simultaneous change in the underlying feature specifications of the TRAP vowel. This results in a phonology that allows for the TRAP and DRESS vowels, in particular, to undergo the influence of the phonetic principles of dispersion and focalization, which lead to lowering and retraction in the acoustic vowel space. Comparison of data from speakers in Thunder Bay, Ontario, and Cape Breton, Nova Scotia, lends support to this hypothesis because the Cape Breton data reveals evidence of two concurrent phonological systems, despite no evidence of change over apparent time. Some Cape Breton speakers display the Ontario (i.e., inland Canada) Canadian Shifted vowel system, while others display a system that bears much greater resemblance to the Eastern New England non-shift dialect, where PALM merges with TRAP instead of LOT-THOUGHT. The current analysis thus predicts that the Canadian Shift or a similar change to the TRAP, DRESS, and KIT vowels will occur in any North American dialect where the PALM-LOT-THOUGHT merger occurs, unless an intervening phonological change alters the contrasts within the phonological system